Draft

OGC Standard

2023 Open Standards Code Sprint Summary Engineering Report
Gobe Hobona Editor Joana Simoes Editor
Version: 1.0
Additional Formats: XML PDF DOC
OGC Standard

Draft

Document number:23-059
Document type:OGC Standard
Document subtype:Implementation
Document stage:Draft
Document language:English

License Agreement

Use of this document is subject to the license agreement at https://www.ogc.org/license

Suggested additions, changes and comments on this document are welcome and encouraged. Such suggestions may be submitted using the online change request form on OGC web site: http://ogc.standardstracker.org/



I.  Executive Summary

TBA

The sprint participants made the following recommendations for future work:

II.  Keywords

The following are keywords to be used by search engines and document catalogues.

ogcdoc, OGC document, API, openapi, html

III.  Security considerations

No security considerations have been made for this document.

IV.  Submitting Organizations

The following organizations submitted this Document to the Open Geospatial Consortium (OGC):

V.  Submitters

All questions regarding this document should be directed to the editor or the contributors:

NameOrganizationRole
Gobe HobonaOpen Geospatial ConsortiumEditor
Joana SimoesOpen Geospatial ConsortiumEditor
Tom KralidisMeteorological Service of CanadaEditor
Andreas MatheusSecure DimensionsContributor
Paul van GenuchtenISRIC- World soil informationContributor
Iván Sánchez OrtegaOSGeoContributor
Florent GravinCamptocampContributor
Clemens Porteleinteractive instruments GmbHContributor
Jérôme Jacovella-St-LouisEcere CorporationContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor
TBATBAContributor

VI.  Abstract

The subject of this Engineering Report (ER) is a code sprint that was held from October 30th to November 1st, 2023 to advance the support and development of open standards within the developer community. The code sprint was organized by the Open Geospatial Consortium (OGC) and hosted by Geovation in London, England. The code sprint was sponsored by Ordnance Survey (OS), the European Union Satellite Centre (SatCen), US National Geospatial Intelligence Agency (NGA), and the UK Defence Science and Technology Laboratory (Dstl).

1.  Scope

This Engineering Report (ER) presents the high level architecture of the code sprint and describes each of the standards and software packages that were deployed in support of the code sprint. The ER also discusses the results and presents a set of conclusions and recommendations.

2.  Normative references

The following documents are referred to in the text in such a way that some or all of their content constitutes requirements of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the referenced document (including any amendments) applies.

Clemens Portele, Panagiotis (Peter) A. Vretanos, Charles Heazel: OGC 17-069r4, OGC API — Features — Part 1: Core corrigendum. Open Geospatial Consortium (2022). https://docs.ogc.org/is/17-069r4/17-069r4.html.

Mark Burgoyne, David Blodgett, Charles Heazel, Chris Little: OGC 19-086r5, OGC API — Environmental Data Retrieval Standard. Open Geospatial Consortium (2022). https://docs.ogc.org/is/19-086r5/19-086r5.html.

Benjamin Pross, Panagiotis (Peter) A. Vretanos: OGC 18-062r2, OGC API — Processes — Part 1: Core. Open Geospatial Consortium (2021). https://docs.ogc.org/is/18-062r2/18-062r2.html.

Joan Masó, Jérôme Jacovella-St-Louis: OGC 20-057, OGC API — Tiles — Part 1: Core. Open Geospatial Consortium (2022). https://docs.ogc.org/is/20-057/20-057.html.

Charles Heazel: OGC 19-072, OGC API — Common — Part 1: Core. Open Geospatial Consortium (2023). https://docs.ogc.org/is/19-072/19-072.html.

3.  Terms, definitions and abbreviated terms

This document uses the terms defined in OGC Policy Directive 49, which is based on the ISO/IEC Directives, Part 2, Rules for the structure and drafting of International Standards. In particular, the word “shall” (not “must”) is the verb form used to indicate a requirement to be strictly followed to conform to this document and OGC documents do not use the equivalent phrases in the ISO/IEC Directives, Part 2.

This document also uses terms defined in the OGC Standard for Modular specifications (OGC 08-131r3), also known as the ‘ModSpec’. The definitions of terms such as standard, specification, requirement, and conformance test are provided in the ModSpec.

For the purposes of this document, the following additional terms and definitions apply.

3.1. API

An Application Programming Interface (API) is a standard set of documented and supported functions and procedures that expose the capabilities or data of an operating system, application, or service to other applications (adapted from ISO/IEC TR 13066-2:2016).

3.2. coordinate reference system

A coordinate system that is related to the real world by a datum term name (source: ISO 19111).

3.3. OpenAPI Document

A document (or set of documents) that defines or describes an API. An OpenAPI definition uses and conforms to the OpenAPI Specification (https://www.openapis.org).

3.4. Web API

An API using an architectural style that is founded on the technologies of the Web [source: OGC API — Features — Part 1: Core].

3.5.  Abbreviated terms

API

Application Programming Interface

CITE

Compliance Interoperability & Testing Evaluation

CRS

Coordinate Reference System

EDR

Environmental Data Retrieval

GIMI

GEOINT Imagery Media for ISR (GIMI)

GIS

Geographic Information System

MUDDI

Model for Underground Data Definition and Integration

NGIIS

Next Generation ISR Imagery Standards

OGC

Open Geospatial Consortium

OWS

OGC Web Services

REST

Representational State Transfer

TEAM

Test, Evaluation, And Measurement Engine

4.  High-Level Architecture

The focus of the sprint was on the support of implementations of open geospatial standards across various open source software projects. Implementations of approved and candidate OGC Standards were deployed in participants’ own infrastructure in order to build a solution with the architecture shown below in Figure 1. As illustrated, the sprint architecture was designed to enable client applications to connect to different servers that implement open geospatial standards. The architecture also included several different software libraries that support open geospatial standards and enable the extraction, transformation, and loading of geospatial data.

Figure 1 — High Level Overview of the Sprint Architecture

The rest of this section describes the software deployed and standards implemented during the code sprint.

4.1.  Approved OGC Standards

4.1.1.  OGC SensorThings API

The OGC SensorThings API Standard provides an open and harmonized way to interconnect devices, data, and applications over the web and on the Internet of Things (IoT). At a high level the OGC SensorThings API provides two main parts, namely Part I — Sensing, and Part II — Tasking. The Sensing part provides a standard way to manage and retrieve observations and metadata from a variety of IoT sensor systems. The Tasking part provides a standard way for tasking IoT devices, such as sensors or actuators. The SensorThings API follows REST principles and uses JSON for encoding messages, as well as MQTT for publish/subscribe operations.

4.1.2.  OGC STAplus — an extension of the OGC SensorThings API

STAplus is an extension to the OGC SensorThings data model. Inspired by Citizen Science applications, STAplus supports the ‘ownership concept’ whereby observations collected by sensors are owned by (different) users that may express the license for re-use. In addition to the ownership and license abilities, the extension specified by STAplus supports the expression of explicit relations between observations and the creation of group(s) of observations.

4.1.3.  OGC API — Features

The OGC API — Features Standard offers the capability to create, modify, and query spatial data on the Web. The Standard specifies requirements and recommendations for APIs that want to follow a standard way of sharing feature data. The specification is a multi-part standard. Part 1, labelled the Core, describes the mandatory capabilities that every implementing service has to support and is restricted to read-access to spatial data that is referenced to the World Geodetic System 1984 (WGS 84) Coordinate Reference System (CRS). Part 2 enables the use of different CRS, in addition to the WGS 84. Additional capabilities that address specific needs will be specified in additional parts. Envisaged future capabilities include, for example, support for creating and modifying data, more complex data models, and richer queries.

The OGC API — Features Standard is part of the OGC API family of standards. OGC API Standards define modular API building blocks to spatially enable Web APIs in a consistent way. The standards make use of the OpenAPI specification for defining the API building blocks.

4.1.4.  OGC API — Tiles

OGC API — Tiles specifies a Standard for Web APIs that provide tiles of geospatial information. The Standard supports different forms of geospatial information, such as tiles of vector features (“vector tiles”), coverages, maps (or imagery) and potentially eventually additional types of tiles of geospatial information.

Vector data represents geospatial objects such as points, lines, and polygons. Tiles of vector feature data (i.e., Vector Tiles) represent partitions of vector data covering a large area (e.g., lines representing rivers in a country).

In this context, a map is essentially an image representing at least one type of geospatial information. Tiles of maps (i.e., Map Tiles) represent subsets of maps covering a large area (e.g., a satellite image).

4.1.5.  OGC API — Coverages

The OGC API — Coverages candidate standard establishes an access mechanism for coverages as defined by the OGC Abstract Specification Topic 6 / ISO 19123 Schema for coverage geometry and functions through a Web API which can be described by an API description language such as the OpenAPI specification. Arguably the most popular type of coverage is that of a gridded coverage. Gridded coverages have a grid as their domain set describing the direct positions in multi-dimensional coordinate space, depending on the type of grid. Satellite imagery is typically modeled as a gridded coverage, for example.

4.1.6.  OGC API — Environmental Data Retrieval

The OGC API — Environmental Data Retrieval (EDR) Standard API provides a family of lightweight interfaces to access Environmental Data resources. Each resource addressed by an EDR API maps to a defined query pattern. This specification identifies resources, captures compliance classes, and specifies requirements which are applicable to OGC Environmental Data Retrieval API’s. This specification addresses two fundamental operations; discovery and query. Discovery operations allow the API to be interrogated to determine its capabilities and retrieve information (metadata) about this distribution of a resource. This includes the API definition of the server as well as metadata about the Environmental Data resources provided by the server. Query operations allow Environmental Data resources to be retrieved from the underlying data store based upon simple selection criteria, defined by this standard and selected by the client.

Recently, the EDR SWG has commenced work on EDR Part 2: Publish-Subscribe workflow.

4.1.7.  OGC GeoPose

OGC GeoPose is a standard for exchanging the location and orientation of real or virtual geometric objects (Poses) within reference frames anchored to the earth’s surface (Geo) or within other astronomical coordinate systems. GeoPose specifies a single encoding in JSON format (IETF RFC 8259). The use cases addressed by GeoPose involve interactions between information systems or between an information system and a storage medium. The role of a GeoPose is to convey the orientation and position of a real or virtual object.

4.2.  Candidate OGC Standards

4.2.1.  Model for Underground Data Definition and Integration (MUDDI)

The MUDDI candidate standard serves as a framework to make datasets that utilize different information for underground objects interoperable, exchangeable, and more easily manageable.

The MUDDI Conceptual Model represents real-world objects found underground. It was designed as a common basis to create implementations that make different types of subsurface data — such as those relating to utilities, transport infrastructure, soils, ground water, or environmental parameters — interoperable in support of a variety of use cases and in different jurisdictions and user communities.

4.2.2.  OGC API — Records

The OGC API — Records candidate Standard provides discovery and access to metadata records about resources such as features, coverages, tiles / maps, models, assets, services or widgets. The candidate Standard enables the discovery of geospatial resources by standardizing the way collections of descriptive information about the resources (metadata) are exposed. The candidate Standard also enables the discovery and sharing of related resources that may be referenced from geospatial resources or their metadata by standardizing the way all kinds of records are exposed and managed.

4.2.3.  OGC Publish/Subscribe extension of OGC API Standards

OGC APIs provide Web based capabilities which are typically based on polling for collection resource updates (new features/records items, coverages, maps, etc.). Depending on a collection’s temporal resolution or frequency of updates, an event-driven / Publish-Subscribe architecture provides a time, efficient and low latency approach for delivery of data updates.

4.3.  Specifications from the Community

4.3.1.  NGIIS GIMI

TBA

4.3.2.  AsyncAPI

AsyncAPI is a specification by the Linux Foundation for describing and documenting message-driven APIs in a machine-readable and protocol-agnostic format. The specification defines a set of files required to describe a message-driven API. The files describing an API in accordance with the AsyncAPI Specification may be encoded in JSON or YAML.

4.4.  Software Projects and Products

4.4.1.  OSGeo GeoNetwork

GeoNetwork is a catalog application for managing spatially referenced resources. It provides metadata editing and search functions as well as an interactive web map viewer.

4.4.2.  geo-heif-viewer

Then geo-heif-viewer is a proof-of-concept viewer for HEIF files containing geographical data (according to the GIMI profile), e.g. satellite imagery.

This viewer uses custom-built versions of:

…​as well as an unmodified copy of:

4.4.3.  OSGeo pygeoapi

pygeoapi is a Python server implementation of the OGC API suite of Standards. The project emerged as part of the next generation OGC API efforts in 2018 and provides the capability for organizations to deploy a RESTful OGC API endpoint using OpenAPI, GeoJSON, and HTML. pygeoapi is open source and released under an MIT license. pygeoapi is an official OSGeo Project as well as an OGC Reference Implementation.

pygeoapi supports numerous OGC API Standards. The official documentation provides an overview of all supported standards.

4.4.4.  OSGeo pycsw

pycsw is an implementation of OGC API — Records and the OGC’s Catalogue Service for the Web (CSW) written in Python. Started in 2010 (more formally announced in 2011), pycsw allows for the publishing and discovery of geospatial metadata via numerous APIs (CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU), providing a standards-based metadata and catalogue component of spatial data infrastructures. pycsw is Open Source, released under an MIT license, and runs on all major platforms (Windows, Linux, Mac OS X). pycsw is an official OSGeo Project as well as an OGC Reference Implementation.

pycsw supports numerous metadata content and API standards, including OGC API — Records — Part 1.0: Core and its associated specifications. The official documentation provides an overview of all supported standards.

4.4.5.  ldproxy

ldproxy is an implementation of the OGC API family of standards, available under the MPL 2.0 open source license. ldproxy is developed by interactive instruments GmbH, written in Java, and is deployed using Docker containers. ldproxy implements all parts of OGC API — Features, OGC API — Tiles, OGC API — Styles, OGC API — 3D GeoVolumes, and OGC API — Routes. It is an OGC Reference Implementation for Parts 1 and 2 of OGC API — Features.

4.4.6.  MariaDB CubeWerx CubeServ

The CubeWerx server (“cubeserv”) is implemented in C and currently implements the following OGC Standards and draft specifications.

  • Multiple conformance classes and recommendations of the OGC API — Tiles — Part 1: Core Standard.

  • Multiple conformance classes and recommendations of the OGC API — Maps — Part 1: Core candidate Standard.

  • All conformance classes and recommendations of the OGC API — Features — Part 1: Core Standard.

  • Multiple conformance classes and recommendations of the OGC API — Records — Part 1: Core candidate Standard.

  • Multiple conformance classes and recommendations of the OGC API — Coverages — Part 1: Core candidate Standard.

  • Multiple conformance classes and recommendations of the OGC API — Processes — Part 1: Core Standard.

  • Multiple versions of the Web Map Service (WMS), Web Processing Service (WPS), Web Map Tile Service (WMTS) and Web Feature Service (WFS) Standards.

  • A number of other “un-adopted” OGC Web Service draft specifications including the Testbed-12 Web Integration Service, OWS-7 Engineering Report — GeoSynchronization Service, and the Web Object Service prototype.

The cubeserv executable supports a wide variety of back ends including Oracle, MariaDB, SHAPE files, etc. It also supports a wide array of service-dependent output formats (e.g., GML, GeoJSON, Mapbox Vector Tiles, MapMP, etc.) and coordinate reference systems.

4.4.7.  GNOSIS Map Server

The GNOSIS Map Server is written in the eC programming language and supports multiple OGC API Standards. GNOSIS Map Server supports multiple encodings including GNOSIS Map Tiles (which can contain either vector data, gridded coverages, imagery, point clouds or 3D meshes), Mapbox Vector Tiles, GeoJSON, GeoTIFF, GML and MapML. An experimental server is available online at https://maps.gnosis.earth/ogcapi and has been used in multiple OGC Innovation Program initiatives.

5.  Results

The code sprint included multiple software libraries, OWS implementations, OGC API implementations and a variety of client applications. This section presents the key results from the code sprint.

5.1.  Approved and Candidate OGC Standards

5.1.1.  OGC API — Records

The OGC API — Records team held two breakouts which focused on advancing the specification for OAB review.

Discussions included:

  • target of December 2023 for OGC API — Records — Part 1: Core document submission for OAB

  • issue sweep

  • requirements refactoring (shared query parameters)

  • resulting ATS refactoring

  • update on templating links (allowing for using one of variables or varBase)

  • clarification on …​/items q parameter

  • cleanup and validation of example JSON documents; agreement to tidy up YAML schemas and build CI workflow to ensure schemas are always valid, a bundle is built, and examples are schema valid

  • proposal for facets

  • advertising facets and their types (options: separate endpoints, addition of facet: true properrites in (…​/queryables)

  • specifying facets on …​/items requests

  • agreement to review / approve proposal PR and move forward as an extension

  • disussion on federation/distributed search

  • how to address live / distributed search (without metadata harvesting/metadata aggregration)

  • presents issues such as:

  • how to present results (multipart, single document with ‘grouped’ records [would break GeoJSON interoperability])

  • how to sort results

  • this could be accomplished by a front end client (in Python or JavaScript, for example)

  • some organizations want to provide a unified single catalogue that streamlines the search xperience for users

  • could also be put forth as a best practice on implementation

  • way forward is to put forth a proposal for discussion on requirements

5.1.2.  OGC API — Environmental Data Retrieval

5.1.2.1.  Part 1: Core

TBD

5.1.2.2.  Part 2: Publish-Subscribe workflow

The OGC API — Environmental Data Retrieval team and Pub/Sub enthusiasts provided two 5 minute pitch presentations, a mentor session one breakout which focused on Publish-Subscibe workflows for EDR and other OGC API specifications.

The 5 minute pitch presentations focused on Part 1: Core and Part 2: Publish-Subscribe capabilities to help position the subsequent mentor and breakout sessions.

The EDR Pub/Sub mentor session included demonstrations from the joint Open Software and Open Standards Code Sprint between OGC/OSGeo/ASF at Lausanne during April 2023, as well as a demo from NOAA demonstrating Pub/Sub integration with EDR.

A breakout session was also held, focusing on technical details of EDR Part 2.

An ldproxy Pub/Sub demo was given connecting to the WMO WIS2 Global Broker, triggering notification observations into an OGC API Feature collection as well as an additional ldproxy broker. Observations were made available using the OGC API endpoint structure with a prefix for a given server and suffix for a specific observed property. Security was also demonstrated by way of using Open ID Connect.

Discussions included:

  • Links requirements (https://github.com/opengeospatial/ogcapi-environmental-data-retrieval/issues/451#issuecomment-1787420407)

  • adding a channel property to a link object is valuable for efficient binding capabilities for Pub/Sub aware clients

  • non-aware clients would simply be presented a link to a Pub/Sub broker without further context

  • this is what the AsyncAPI document is there for (at the landing page, etc.)

  • we should add the ability to cross-reference collection ids from an AsyncAPI document

  • channels.*.x-ogc-api-collection.url, for example

  • update link requirement to be a permission to facilitate Pub/Sub aware client interoperability

  • done in https://github.com/opengeospatial/ogcapi-environmental-data-retrieval/pull/464

  • Channels requirements (https://github.com/opengeospatial/ogcapi-environmental-data-retrieval/issues/452#issuecomment-1787419817)

  • is the OGC API endpoint topic structure enough?

  • TB19 ldproxy demo prefixed topics with tokens to help make unique, which is valuable in the context of broker message aggregation

  • we could also consider filters as part of a topic

  • could be advanced functionality

  • turn channels requirement into separate conformance class

  • done in https://github.com/opengeospatial/ogcapi-environmental-data-retrieval/pull/465

  • Message payload:

  • should make clear that if the message payload is an item, then id property should persist. Else, if a notificationm, the id would be a GUID.

  • Cross SWG engagement and longer term Pub/Sub SWG activities

  • Pub/Sub is a valuable capability for numerous OGC APIs

  • Connected Systems

  • Features: feature transactions

  • Processes: subscribing to async jobs channels, new processes

  • Records: aggregation/metadata ingest/publication

  • Common: collections publishing data updaetes (e.g. publication of new NWP model run for EDR or Maps

  • engage with SWGs to determine Pub/Sub approach to notify workflow/patterns on changes to resources

A live demo was given which exemplified the capabilities of Pub/Sub as a workflow for metadata catalogue management.

5.1.3.  OGC Features and Geometries JSON

A free and open source json-fg-validator[https://github.com/tomkralidis/json-fg-validator] tool was created as an executable test suite to test JSON FG documents against the ETS requirements. The initial implementation provides a framework to test and report. The first pass covered most of the “Core” Requirements Class.

The tool is implemented in Python and contributions are welcome and encouraged.

Figure 2 — Screenshot of the json-fg-validator tool providing an ETS report.

5.1.4.  Geonovum testing of JSON-FG support in GDAL

Sprint participants from Geonovum worked on testing the JSON-FG candidate Standard and evaluating GDAL’s implementation of the candidate Standard. To facilitate the evaluation, they specified a series of test scenarios that includes tests of transformation of GML to JSON-FG, GeoPackage to JSON-FG and transformation from 3D data formats to JSON-FG. A more detailed discussion of the tests and their results is presented in an associated report [3]. Although the tests did not reveal any bugs nor issues found in GDAL’s support of JSON-FG, there were a number of findings that could inform future versions of GDAL.

The tests were applied to source data that contained features that were referenced to the Dutch coordinate reference system Rijksdriehoeks (RD). During transformation of feature data to JSON-FG, by default GDAL creates a geometry member that has coordinates that a referenced to WGS84. Further, also by default, GDAL creates a place member with coordinates obtained from the source data during transformation of feature data to JSON-FG. Although this conforms with the JSON-FG candidate Standard, it may not be what some users expect.

One of the findings was that when GDAL cannot identify a source property for the temporal information, it fills the time member of the JSON-FG file with a null value. There does not appear to a way to point GDAL at an appropriate source for filling the time member. However, as GDAL looks for an element called time, time_start, or time_end, it is possible to use a GDAL tool called ogr2ogr to remap the properties of the source data to an output property that has a name that GDAL expects (e.g. time). Another finding was that there is currently no apparent option to have GDAL create a JSON-FG file that references a JSON Schema file that describes the featureSchema.

5.1.5.  Example

TBA

5.2.  Specifications from the Community

5.2.1.  Towards open source libraries for GIMI

This section presents findings of the prototyping GIMI support in a selection of open source libraries. The prototyping and assessment was carried out by Brad Hards.

As a community we have an ecosystem that has some key dependencies that include GDAL + libheif, mp4box.js, rasterio, rio-tiler, pygeoapi etc. There are also some commercial libraries that sit alongside these open source libraries. GDAL is used by several applications such as GDAL. Rasterio is a very import tool. So the combination of tools and libraries presents a big stack is illustrated in Figure TBA. The sprint participants sought to show how this stack could be used to support GIMI.

Figure 3

Figure TBA shows a screenshot of an unmodified version of QGIS displaying two GIMI test files, overlayed on top of OpenStreetMap.

Figure 4

The sprint participants created a Jupyter Notebook, shown in Figure TBA, that used many of the aforementioned libraries to perform some data science analysis on GIMI imagery files.

Figure 5

Further work on pygeoapi involved importing a GIMI file into pygeoapi for serving through an interface conforming to OGC API – Coverages. This exposed coverage metadata as illustrated in Figure TBA.

Figure 6

Using the open source TiTiler toolkit, the sprint participants were able to partition GIMI test files into a tileset and to provision a pygeoapi instance with the tileset. This made it possible for the imagery to be accessible through an interface conforming to OGC API – Tiles and thus be displayed on an instance of OpenLayers.

Figure 7

In conclusion, the sprint participants found that there are a number of base libraries that are needed to improve the migration path for several applications. Amongst those base libraries are GDAL and libheif. Also needed are more tools, more examples of test data, and validators to facilitate the evaluation of test files.

5.2.2.  Example 2

TBA

5.3.  Software Projects and Products

5.3.1.  ldproxy

Sprint participants from interactive instruments worked on ldproxy’s support of a Publish/Subscribe (PubSub) capability for OGC API Standards. This included creation of a resource using the Topographic Data Store (TDS) schema of the NGA and enabling the resource to support PubSub as well as Create, Replace, Update, and Delete (CRUD) operations. The CRUD capabilities are described in an OpenAPI definition document and the PubSub capabilities are described in an AsyncAPI definition document. Links to both the OpenAPI and AsyncAPI definition documents are clearly visible on the landing page, as shown in Figure TBA.

Figure 8

Below is a fragment of the AsyncAPI definition document created to describe the supported PubSub capability. The fragment shows two of the thirty channels created for monitoring changes on feature collections provided by the API.

{
 
"info" : {
   
"title" : "Daraa",
   
"description" : "This is a test dataset for the Open Portrayal Framework thread in the OGC Testbed-15 as well as for the OGC Vector Tiles Pilot Phase 2. The data is OpenStreetMap data from the region of Daraa, Syria, converted to the Topographic Data Store schema of NGA.",
   
"version" : "1.0.0",
   
"contact" : {
     
"name" : "Clemens Portele, interactive instruments GmbH",
     
"email" : "portele@interactive-instruments.de"
   
},
   
"license" : {
     
"name" : "The dataset was provided by the US National Geospatial Intelligence Agency (NGA) for development and testing. For any reuse of the data outside this API, please contact NGA."
   
}
 
},
 
"servers" : {
   
"t19" : {
     
"protocol" : "secure-mqtt",
     
"protocolVersion" : "3.1.1",
     
"url" : "t19.ldproxy.net:8883",
     
"bindings" : {
       
"clientId" : "cs2310.ldproxy.net",
       
"cleanSession" : true,
       
"bindingVersion" : "0.1.0"
     
}
   
}
 
},
 
"channels" : {
   
"ogcapi/cs2310.ldproxy.net/daraa/collections/AeronauticCrv/items" : {
     
"subscribe" : {
       
"operationId" : "featureChange_AeronauticCrv_items",
       
"bindings" : {
         
"qos" : 0,
         
"retain" : false,
         
"bindingVersion" : "0.1.0"
       
},
       
"message" : {
         
"$ref" : "#/components/messages/featureChange_AeronauticCrv"
       
}
     
},
     
"servers" : [ "t19" ]
   
},
   
"ogcapi/cs2310.ldproxy.net/daraa/collections/AeronauticSrf/items" : {
     
"subscribe" : {
       
"operationId" : "featureChange_AeronauticSrf_items",
       
"bindings" : {
         
"qos" : 0,
         
"retain" : false,
         
"bindingVersion" : "0.1.0"
       
},
       
"message" : {
         
"$ref" : "#/components/messages/featureChange_AeronauticSrf"
       
}
     
},
     
"servers" : [ "t19" ]
   
},

}

The team from interactive instruments also set up a broker for experimenting with the messages. The ldproxy instance offers access to approximately 30 different feature collections. Whenever something is created or changed in a feature collection, a message is sent to the broker. A screenshot showing the channels being monitored through an application called MQTT Explorer is presented in Figure TBA.

Figure 9

5.3.2.  OSGeo pycsw

The pycsw project upgraded support for the OGC API — Records q parameter based on the latest updates (PRs in https://github.com/geopython/pycsw/pull/912 https://github.com/geopython/pycsw/pull/913). In addition, record parsing and ingest updates and fixes were implemented given specification / schema updates.

5.3.3.  OSGeo pygeoapi

5.3.3.1.  Implementation of a CSW Facade

The pygeoapi project implemented an OGC API — Records CSW Facade plugin as a means to bridge CSW implementations. Note that support is bound to CSW 2/3 and the CSW Record core model (ISO 19115/19139 support can be implemented as future work). This work was merged to the core pygeoapi codebase.

Figure 10 — Screenshot of the pygeoapi OGC API - Records provider architecture for the CSW Facade.

A live demonstration was provide using the WMO World Ozone and Ultraviolet Radiation Data Centre (WOUDC) CSW server.

Figure 11 — Screenshot of the pygeoapi OGC API - Records provider for the CSW Facade.

5.3.3.2.  Support for Publish/Subscribe operations

A live demo was given which exemplified the capabilities of Pub/Sub as a workflow for metadata catalogue management, using the WMO WIS2 Global Discovery Catalogue Reference Implementation (powered by pygeoapi). In this demo, the catalogue is subscribed to an MQTT broker which is publishing messages of new discovery metadata records. On notification of new metadata, the catalogue is triggered to validate and ingest the metadata into a metadata collection, thereby making it available via OGC API — Records.

Figure 12 — Screenshot of MQTT Explorer with metadata notification updates.

Figure 13 — Screenshot of metadata published to OGC API - Records using pygeoapi.

5.3.3.3.  Support for the draft GIMI profile

Sprint participants from Geobeyond worked on the GIMI thread of the code sprint. Their goal was to implement the ability to serve files using OGC API – Coverages and OGC API – Tiles capabilities offered by pygeoapi. An additional goal was to enable a worfkflow that supports modification of the files. The libraries that they modified or implemented for this work are available on the gimirrai repository on GitHub.

As an initial objective, they set out to see whether the libheif library could be enabled to support reading and writing of GIMI files through GDAL. Their expectation was that they could use a library such as rasterio to further support this capability. Upon testing libheif, they found that libheif does not support some of the needed capabilities. Popular raster-handling tools such as GDAL are not by default able to support GIMI files. Some modification is required to enable such tools to support for the format.

With help from Brad Hards, they compiled a version of libheif that could read GIMI files. They then compiled GDAL, pillow-heif and rasterio to enable the reading of GIMI files through Python.

They were also able to decode and parse the GIMI KLV bytestream through pillow-heif. This enabled them to review the geographic bounds declared in the metadata of the GIMI files. Figure TBA shows the geographic bounds of a sample GIMI file displayed on a pygeoapi interface. Afterwards, they sought to enable pygeoapi to support the reading of GIMI files. Enabling pygeoapi to support the reading of GIMI files, would make it possible to serve the imagery files through interfaces conforming to OGC API Standards.

Figure 14

For OGC API – Coverages, they were able to write a custom provider on top of an existing pygeoapi provider. All that was needed was to modify the parts related to providing metadata.

For the OGC API Tiles implementation, it was a bit more involved because while for OGC API — Coverages they were working with single GIMI files, for OGC API — Tiles it involved having to partition an image into multiple tiles and then to georeference each tile. For this they needed to check whether the GDAL VRT (https://gdal.org/drivers/raster/vrt.html) capability could support this. They were able to leverage the GDAL VRT provider which supports Ground Control Points (GCPs). Figure TBA shows a screenshot of a GIMI file displayed on a web map served by pygeoapi.

Figure 15

However, within the duration of the code sprint, it was not possible to fully implement transaction support in pygeoapi. This was partly because of limitations of pillow-heif in the modification of pre-existing GIMI files.

In conclusion, it is currently not very easy to work with GIMI files at the Python level. Ideally proper support would be implemented in lower-level libraries first namely libheif and GDAL. With lower-level support in place, usage in Python would become transparent. These lower-level libraries could then be combined with applications and libraries such as pygeoapi, rasterio, and rio-tiler.

5.3.4.  Safe Software FME

5.3.4.1.  MUDDI

The sprint participants from Safe Software obtained Sanitary system from the City of Burnaby in Canada for use as input data in the code sprint. The data is available from the City as open data (https://data.burnaby.ca/datasets/burnaby::sanitary-main-/explore). The data includes information about the Sanitary system and connectors, as well as some manhole covers. Using the FME software product, the team sought to try out the MUDDI application schema that had been provided by interactive instruments in support of the code sprint. FME Desktop is a geospatial extract, transformation and load software platform. The 2021 version of FME Desktop is Certified OGC Compliant to GML 3.2.1, GeoPackage 1.2 and KML 2.2 (https://www.ogc.org/resources/product-details/?pid=1681).

The workflow for converting from plain GeoJSON to MUDDI GML is illustrated in Figure 16. The workflow takes source data and copies attributes across to output feature types. Excess attribution that is not required in the output feature type can be thrown away. A Network feature acts as a parent to all of the other associated features. Therefore, the mains and service lines are associated with a network. This includes fittings, mains and service lines to which some date fields are added. The workflow also creates a service area which represents the bounding box around the features.

Figure 16 — Screenshot of workflow for converting from plain GeoJSON to MUDDI GML

A summary of the workflow is below:

  1. Read GeoJSON

  2. Generate and convert timestampts to ISO format

  3. Map attributes from source to MUDDI schema

  4. Create missing attributes, including all required attributes

  5. Create parent network features

  6. Generate gml_ids from UUIDs and gml_parent_ids to associate network elements with parent networkds

  7. Set geometry property

  8. Write MUDDI GML using ApplicationSchema.xsd from Interactive Instruments.

The workflow invokes the GML writer capability of FME. An overview of the GML writer’s schema capability is available on the Safe Software website (https://community.safe.com/s/article/gml-writing-with-application-schemas).

The sprint participants reported that they encountered an issue whereby some of the required attribution was not available from the source dataset. Fortunately, in such situations, the software logged an error message that listed the required attributes that had not been field.

The FME Desktop application was able to write to MUDDI GML and then to read it back, as illustrated in Figure 17. From the screenshot it can be seen on the top left that there were 11,000 fittings. The screenshot also shows the two networks. Those networks are associated with service lines. One of the service lines is associated with names. The Network covariance link is the actual line. The Service Area is represented by a bounding box.

Figure 17 — Screenshot of MUDDI GML data displayed in FME

On the right-hand side of Figure 17 is shown the attribution from the data. Most of the attribution is carried through from the source data, however for illustrative purposes in this Code Sprint some attribution is synthetic. The workflow also added an elevation dimension to the geometries, which was taken from another feature which had up and down elevation feature properties.

5.3.4.2.  CityGML

Sprint participants from Safe Software also worked on a demonstration of CityGML2 to CityGML3 conversion of Montreal data. All attribution (apart from appearances) was pulled across. An illustration of the workflow for converting from CityGML2 to CityGML3 is shown in Figure 18.

Figure 18 — Screenshot of workflow for converting from CityGML2 to CityGML3

A separate workflow was created for converting from IFC to CityGML3. An illustration of the workflow is shown in Figure 19.

Figure 19 — Screenshot of workflow for converting from IFC to CityGML3

The output CityGML is presented in Figure 20. The figure shows rooms in a building, with one of the rooms selected. The attributes of the selected room, which are presented on the right-hand side, report that the room is a Living Room.

Figure 20 — Screenshot of the output CityGML data displayed in FME

In summary, sprint participants from Safe Software were able to write to MUDDI GML using the application schemas provided by interactive instruments. The workflows for writing to MUDDI GML captured multiple two-dimensional (2D) and three-dimensional (3D) geometry types (points, lines, polygons). Topology was also populated with xlink_hrefs.

The sprint participants from Safe Software also started experimenting with the CityGML3 Dynamizer to explore how it might work in future versions of the workflow. They successfully converted CityGML2 data to CityGML3 data, as well as IFC data to CityGML3 data. They were able to store IFC spaces as rooms and support a range of other new CityGML3 feature types and properties. They also confirmed that they could read the output back with the new CityGML3 reader.

5.3.5.  Example 3

TBA

6.  Discussion

6.1.  Example

TBA

6.2.  Example 2

TBA

6.3.  Example 3

TBA

7.  Conclusions

TBA

7.1.  Future Work

The sprint participants made the following recommendations for future work items:

  • TBA

  • TBA

  • TBA


Annex A
(informative)
Revision History

DateReleaseAuthorPrimary clauses modifiedDescription
2023-10-200.1G. Hobonaallinitial version

Bibliography

[1]  OGC: OGC 21-008: Joint OGC OSGeo ASF Code Sprint 2021 Summary Engineering Report, 2021

[2]  Aleksandar Balaban: OGC 21-019, OGC Testbed-17: Attracting Developers: Lowering the entry barrier for implementing OGC Web APIs. Open Geospatial Consortium (2022). https://docs.ogc.org/per/21-019.html

[3]  Linda van den Brink, Arnoud de Boer: Testing OGC JSON-FG Draft Specification and its GDAL-DEV Implementation. Geonovum (2023). https://geonovum-labs.github.io/test-ogc-json-fg